Moving Object Detection Apparatus And Method

ABSTRACT

Disclosed is directed to a moving object detection apparatus and method. The apparatus comprises an image capture module, an image alignment module, a temporal differencing module, a distance transform module, and a background subtraction module. The image capture module derives a plurality of images in a time series. The image alignment module aligns the images if the image capture module is situated on a movable platform. The temporal differencing module performs temporal differencing on the captured images or the aligned images, and generates a difference image. The distance transform module transforms the difference image into a distance map. The background subtraction module applies the distance map to background subtraction technology and compares the results with the current captured image, so as to obtain the information for moving objects.

FIELD OF THE INVENTION

The present invention generally relates to a moving object detectionapparatus and method.

BACKGROUND OF THE INVENTION

Moving object detection plays an important role in automaticsurveillance systems. Surveillance systems detect abnormal securityevents by analyzing the trajectory and behavior of moving objects in animage, and notify the related security staff. The development of thesecurity robots moves towards the intelligent security robots withabnormal event detection capability to support dynamic deployment andrepetitive, continuous surveillance. The moving object detection aims toreplace the passive recording widely used in conventional surveillancesystems.

For example, US. Pat. No. 6,867,799 disclosed a method and apparatus forobject surveillance with a movable camera, including the construction ofa surveillance mechanism of maintaining a moving object of interestwithin the filed of view of a movable camera in an object surveillancesystem. According to the selected object of interest, the cameramovement commands are created so that the object of interest remains inthe field of the view of the camera. U.S. Pat. No. 7,123,745 disclosed amethod and apparatus for detecting moving objects in video conferencingand other application. From the continuous video images of a fixedcamera, the difference image technique is used to detect moving personand the position and the size of the head of the person are identified.

U.S. Pat. No. 5,991,428 disclosed a moving object detection apparatusand method, including a foreground moving object detection techniqueapplicable to movable camera. By image segmentation, template matchingand evaluation and voting, the disclosed patent estimates the movingvector of the corresponding areas of the neighboring images. Based onthe dominant moving vector of the image, the align vector between theneighboring images is determined. Based on the align vector, one of thetwo neighboring images is shifted for alignment and differencecomparison to identify the moving object area. U.S. Pat. No. 5,473,364disclosed a video technique for indicating moving objects from a movableplatform. Assuming that the images captured by the front and rearcameras at two consecutive times have only a slight difference, thedisclosed patent aligns the images from the front camera and subtractsfrom the image from the rear camera, and then uses Gaussian pyramidconstruction to compute the area energy to detect the moving objects andobtains more stable moving object profile.

However, image-based moving object detection technique deployed on afixed camera usually cannot provide dynamic security support. In arestricted surveillance area, the surveillance is often ineffective. Onthe other hand, for movable camera surveillance, the movement of thecamera will cause the entire image change and the compensation to theerror caused by the camera movement makes it difficult to use a singleimage-based technique to effectively detect moving objects.

FIGS. 1 and 2 show the moving object detection methods, which integratebackground subtraction and consecutive image difference, proposed byDesa and Spagnolo in 2004 and 2006 respectively. The backgroundsubtraction is to consider the background of an area in foregrounddetection, and the consecutive image difference is to find thedifference in a plurality of consecutive images to detect moving parts.However, in the techniques depicted in FIGS. 1 and 2, the backgroundsubtraction and consecutive image difference are solely integratedcomputationally. Therefore, only the outer profile of moving objects canbe detected, while the inner area of the entire moving objects cannot bedetected.

SUMMARY OF THE INVENTION

The disclosed exemplary embodiments according to the present inventionmay provide an apparatus and method for detecting moving objects. Theinformation of the detected moving objects at least includes the regionwhere the moving objects occur.

In an exemplary embodiment, the disclosed is directed to a moving objectdetection apparatus, comprising: an image capture module, an imagealignment module, a temporal differencing module, a distance transformmodule, and a background subtraction module. The image capture modulederives a plurality of images in a time series. The image alignmentmodule aligns the images if the image capture module is situated on amovable platform. The temporal differencing module performs temporaldifferencing on the captured images or the aligned images, and generatesa difference image. The distance transform module transforms thedifference image into a distance map. The background subtraction moduleapplies the distance map to background subtraction technology andcompares the results with the current captured image, so as to obtainthe information for moving objects.

In another exemplary embodiment, the disclosed is directed to a movingobject detection method, comprising: capturing images at differenttimes; aligning the images at different times if on a movable platform;applying temporal differencing on captured or aligned images to generatea difference image; transforming the difference image into a distancemap; and applying the distance map to the background subtractiontechnology and comparing the results with an current captured image toobtain the moving object information.

The disclosed exemplary embodiments according to the present inventionmay be applied to a platform with a movable camera for detecting movingobjects in real-time. By using temporal differencing to obtain distancemap for enhancing the background subtraction technique, the presentinvention is also applicable to a fixed camera platform to improve thereliability of moving object detection.

The foregoing and other features, aspects and advantages of the presentinvention will become better understood from a careful reading of adetailed description provided herein below with appropriate reference tothe accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an exemplary schematic view of combining backgroundsubtraction and temporal differencing for moving object detection.

FIG. 2 shows another exemplary schematic view of combining backgroundsubtraction and temporal differencing for moving object detection.

FIG. 3 shows a schematic view of an exemplary moving object detectionapparatus, consistent with certain disclosed embodiments.

FIG. 4 shows an exemplary flowchart of a moving object detection method,consistent with certain disclosed embodiments.

FIG. 5 shows an exemplary schematic view of performing image alignment,consistent with certain disclosed embodiments.

FIG. 6 shows an exemplary schematic view of performing temporaldifferencing, consistent with certain disclosed embodiments.

FIG. 7 shows an exemplary schematic view of performing distancetransform, consistent with certain disclosed embodiments.

FIG. 8 shows an exemplary view of performing background subtraction,consistent with certain disclosed embodiments.

FIG. 9 shows an exemplary schematic view of integrating FIG. 5-FIG. 8,consistent with certain disclosed embodiments.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the disclosed exemplary embodiments of the present invention, theimages captured by the movable camera are first compensated by the imageanalysis for the background changed caused by the camera movement. Then,the temporal differencing, distance transformation, and backgroundsubtraction techniques are combined to detect the moving object regionsin a stable manner.

FIG. 3 shows a schematic view of an exemplary moving object detectionapparatus, consistent with certain disclosed embodiments. Referring toFIG. 3, the exemplary moving object detection apparatus 300 may comprisean image capture module 301, an image alignment module 303, a temporaldifferencing module 305, a distance transform module 307, and abackground subtraction module 309.

Image capture module 301 captures images for moving objects 310 atdifferent times. If image capture module 301 is on a movable platform,image align module 303 aligns the images captured at different times.The aligned images are marked as 303 a. Temporal differencing module 305performs temporal differencing on the captured or aligned images toobtain a difference image 305 a. Distance transform module 307transforms difference image 305 a into a distance map 307 a. Backgroundsubtraction module 309 applies distance map 307 a to a backgroundsubtraction technology and compares to the current captured image toobtain a final detection result for moving objects; that is, movingobject information 309 a.

In the case of a movable platform, image align module 303 may providealigned images 303 a to temporal differencing module 305 for referenceand provide alignment parameters to background subtraction module 309for reference. In the case of a static platform, no image alignment isrequired. Therefore, on a static platform, moving object detectionapparatus 300 need not include an image align module 303, and backgroundsubtraction module 309 does not require alignment parameters for input.

FIG. 4 shows an exemplary flowchart of a moving object detection method,according to the exemplary apparatus of FIG. 3, consistent with certaindisclosed embodiments. Referring to FIG. 4, step 410 is to captureimages for moving objects at different times. If image capture module301 is on a moveable platform, step 420 is to align images 301 acaptured at different times to obtain an aligned image 303 a. In step430, temporal differencing technology is performed on aligned image 303a to obtain difference image 305 a. On the other hand, if image capturemodule 301 is on a static platform, step 430 is to perform differencingtechnology directly on captured images at different times to obtaindifference image 305 a. Therefore, as shown in step 430, temporaldifferencing technology is performed on captured images 301 a or alignedimages 303 a to obtain difference image 305 a.

In step 440, distance transform is performed on difference image 305 ainto a distance map 307 a. The distance map 307 a is applied to thebackground subtraction technology and compared with the current capturedimage to obtain the moving object information, as shown in step 450. Themoving object information may include the marking of the area of themoving object, such as foreground pixels. In step 450, the alignmentparameters are also used to align the background model to the currentcaptured image to obtain the moving object information.

In the exemplary embodiment of FIG. 3, image capture module 301 maycapture a series of continuous images 301 a at different times by acamera on a movable or static platform through moving objects in ascene. According to the background of the continuous images captured atdifferent times, image align module 303 may obtain alignment parametersfor aligning the background of these continuous images captured on amovable platform at different times. As shown in FIG. 5, the alignmentparameters may be obtained from two of the three continuous imagesF_(t−1), F_(t), F_(t+1) captured at times t−1, t, and t+1, and to alignboth images F_(t−1), F_(t+1) to F_(t) to eliminate the image changecaused by the movable camera.

In the disclosed exemplary embodiments, several background compensationtechnologies may be used, for example, multi-resolution estimation ofparametric motion models, which is a technology using Gaussian low-passfilter to establish multi-resolution image pyramid, and then estimatingthe motion parameters between two neighboring images by using the leastmean square error (LMSE) analysis to minimize the difference square oftwo neighboring images on each resolution.

FIG. 6 shows an exemplary schematic view of performing temporaldifferencing, consistent with certain disclosed embodiments. Referringto FIG. 6, after aligning images F_(t−1), F_(t+1) to F_(t), based on thedifference between images F_(t−1), F_(t+1) and F_(t), two framedifferences 610, 620 may be obtained. Using frame differences 610, 620and an AND operation 630, difference image 305 a may be obtained todetect the possible foreground area. That is, temporal differencingmodule 305 may apply the analysis of the three continuous images to thecompensated image to detect the possible foreground of moving objects.

The following shows an example for obtaining the difference image fromthree continuous images F_(t−1), F_(t), F_(t+1). Let X_(i) represent theimage location in a scene, and C(X_(i)) be a representation matrix ofX_(i) that may be multiplied by motion parameter matrix. Then, afterimages F_(t−1), F_(t+1) are aligned to F_(t), two motion parametersA_(t−1), A_(t+1) may be obtained. Using the following equation, twodifference frames FD_(t−1), FD_(t+1) may be obtained:

${{FD}_{k}\left( X_{i} \right)} = \left\{ \begin{matrix}{1,} & {{{abs}\left( {{F_{t}\left( X_{i} \right)} - {F_{k}\left( {X_{i}\; + {{C\left( X_{i} \right)}A_{k}^{- 1}}} \right)}} \right)} < \delta_{1}} \\{0,} & {{otherwise},}\end{matrix} \right.$

Where k=t−1, t, t+1, and δ₁ is a threshold value. “AND” operation isapplied to processing the difference frames FD_(t−1), FD_(t+1) to obtaindifference image FA_(t); i.e.,FA_(t)(X_(i))=FD_(t−1)(X_(i))̂FD_(t+1)(X_(i)).

On the other hand, if the continuous images 301 a are captured on astatic platform, no alignment is necessary. The captured images may beprocessed for difference image 305 a to detect the possible foregroundarea of the moving object.

Distance transform module 307 may apply a distance transform technologyto transform difference image 305 a into a distance map 307 a. Thedistance transform technology, such as the following equation, maytransform difference image FA_(t) into a distance map D_(t):

${{D_{t}\left( X_{i} \right)} = \frac{\min \left( {{{{{FA}_{t}\left( X_{i} \right)} - {{FA}_{t}\left( X_{k} \right)}}},\delta_{2}} \right)}{\delta_{2}}},$

where X_(k) is the foreground point closest to X_(i), and δ₂ is themaximum allowed distance. In other words, each point in distance mapD_(t) is the value of the distance between the point and the closestforeground point divided by the maximum allowed distance. The closer toa foreground the point is, the smaller its value is, which means thatthe point is more likely to belonging to the moving object, and viceversa. FIG. 7 shows an exemplary schematic view of performing distancetransform, consistent with certain disclosed embodiments. Referring toFIG. 7, after distance transform on difference image 705, distance map707 is obtained. Distance map 707 may be seen as a probabilitydistribution of the moving object location. Therefore, from the distancemap, the area in which the moving object in a scene may be seen, and thestability of moving object detection may be improved.

Background subtraction module 309 applies distance map to the backgroundsubtraction technology, and compares with the current capture image toobtain the moving object information. FIG. 8 shows an exemplary view ofperforming background subtraction, consistent with certain disclosedembodiments. Referring to FIG. 8, background subtraction module 309applies alignment parameters 811 to background model 812 to align thebackground to the current captured image. Aligned background model 821and image at time t use distance map, such as 707, as an updating rateto update background model, as shown in 830. Also, aligned backgroundmodel 821 having the weight of distance map 707 is compared with theimage captured at time t to perform foreground detection at time t fordetecting moving objects, such as 809.

In the foreground detection stage, because of the background alignmenterror, a region of background pixels may be used for foregrounddetection. The result of the distance transform of the continuousimages, i.e., distance map D_(t) may be used as an adaptive thresholdvalue for foreground detection. The higher the probability of being inforeground is, the lower the threshold value will be; and vice versa.When the background is updated, distance map D_(t) may be used asadaptive updating rate. The higher the probability of being inforeground is, the background is not updated; and vice versa.

Because background subtraction module 309 uses distance map D_(t) as thebasis of parameter tuning in applying distance map to the backgroundsubtraction technology for foreground detection and background updatingat time t, the obtained moving object information not only includes theouter shape of moving objects, but also the internal area of movingobjects.

FIG. 9 shows an exemplary schematic view of integrating FIG. 5-FIG. 8,consistent with certain disclosed embodiments. From FIG. 9 and theprevious description, the disclosed exemplary embodiments according topresent invention compensates the image shift caused by movableplatform, and combines background subtraction with temporal differencingtechnologies in applying distance map obtained by temporal differencingand distance transform for background subtraction to stably detect theforeground moving object area.

In the disclosed exemplary embodiments of the present invention,temporal differencing technology is used to assist the backgroundsubtraction technology to detect foreground object after compensatingthe background caused by background shift. This achieves the objectiveof using a single camera in effective moving object detection. Thedisclosed exemplary embodiments of the present invention also usesdistance transform technology to transform the temporal differencingresult into a distance map that can be seen as a probabilitydistribution for the current location of objects and may be applied tothe background subtraction as a good weighting function for foregrounddetection and background updating.

In foreground detection, the weight on moving object area is increasedand in background updating, the weight of moving object area is reducedso that the moving object may be detected more easily. In this manner,the present invention may improve the conventional backgroundsubtraction technology to detect moving object more stably. The temporaldifferencing mechanism used in the disclosed exemplary embodiments ofthe present invention not only is applicable to the movable platform,but also to the fixed platform to improve the moving object detectionstability.

Although the present invention has been described with reference to theexemplary embodiments, it will be understood that the invention is notlimited to the details described thereof. Various substitutions andmodifications have been suggested in the foregoing description, andothers will occur to those of ordinary skill in the art. Therefore, allsuch substitutions and modifications are intended to be embraced withinthe scope of the invention as defined in the appended claims.

1. A moving object detection apparatus, comprising: an image capturemodule that captures a plurality of images for one or more movingobjects at different times; an image align module that aligns saidplurality of images captured at different times if said image capturemodule is situated on a movable platform; a temporal differencing modulethat performs temporal differencing on said captured or aligned imagesto obtain a difference image; a distance transform module thattransforms said difference image into a distance map; and a backgroundsubtraction module that applies said distance map to a backgroundsubtraction technology and compares with a current captured image toobtain moving object information.
 2. The apparatus as claimed in claim1, wherein in the case of a movable platform, said image align moduleprovides one or more alignment parameters to said background subtractionmodule to obtain said moving object information.
 3. The apparatus asclaimed in claim 1, wherein in the case of a movable platform, saidimage align module provides said aligned images to said temporaldifferencing module.
 4. The apparatus as claimed in claim 1, whereinsaid moving object information at least includes the area where said oneor more moving objects occur in the image.
 5. The apparatus as claimedin claim 1, wherein said distance map provides a probabilitydistribution of the location for said one or more moving objects.
 6. Theapparatus as claimed in claim 1, said apparatus is applicable to bothmovable platform and fixed platform.
 7. The apparatus as claimed inclaim 6, wherein in the case of a fixed platform, said apparatus doesnot comprise an image align module.
 8. A moving object detection method,comprising the steps of: capturing a plurality of images for one or moremoving objects at different times; aligning said plurality of imagescaptured at different times if said captured images are on a movableplatform; performing temporal differencing on said captured or alignedimages to obtain a difference image; transforming said difference imageinto a distance map; and applying said distance map to a backgroundsubtraction technology and comparing with a current captured image toobtain moving object information.
 9. The method as claimed in claim 8,wherein in the case of a movable platform, at least an alignmentparameter is applied to a pair of said captured images for aligningimages at different times.
 10. The method as claimed in claim 8, whereinsaid background subtraction technology at least include: applying atleast an alignment parameter to a background model for said one or moremoving objects for aligning said background model to a current capturedimage; updating said current captured image into said aligned backgroundmodel by using said distance map as an updating rate for the backgroundmodel updating; and comparing said aligned background model with saidcurrent captured image by taking said distance map as a weightingfunction to perform foreground detection at current time for detectingsaid one or more moving objects.
 11. The method as claimed in claim 8,wherein in the case of movable platform, alignment is performed onneighboring images before temporal differencing to align background areaof said neighboring images for compensating the image change caused bythe movable platform.
 12. The method as claimed in claim 8, wherein saidtemporal differencing technology is to perform an AND operation on twoframe differences obtained from deformation transformation to computesaid difference image.
 13. The method as claimed in claim 8, wherein thearea where said one or more moving objects occur in an image is seenfrom said distance map.
 14. The method as claimed in claim 10, whereinin foreground detection, said distance map is used as an adaptivethreshold for foreground determination.
 15. The method as claimed inclaim 10, wherein in background updating, said distance map is used asan adaptive updating rate.